Various techniques are known in the art for automated contouring and segmentation of computer images as well as generation of three-dimensional surfaces, e.g. from two-dimensional contour data. Typical objects of interest in medical images include organs such as bladder, prostate, kidneys, and many other types of anatomical objects as is well known in the art. Objects of interest in cellular imaging include, for example, cell nucleus, organelles, etc. It will be understood that the techniques disclosed herein are equally applicable to any type of object of interest.
Exemplary techniques for generating and manipulating three-dimensional surfaces are disclosed in U.S. application Ser. No. 11/848,624, entitled “Method and Apparatus for Efficient Three-Dimensional Contouring of Medical Images”, filed Aug. 31, 2007, and published as U.S. Patent Pub. No. 2009-0060299, and U.S. application Ser. No. 12/022,929, entitled “Method and Apparatus for Efficient Automated Re-Contouring of Four-Dimensional Medical Imagery Using Surface Displacement Fields”, filed Jan. 30, 2008, and published as U.S. Patent Pub. No. 2009-0190809, the entire disclosures of each of which are incorporated herein by reference.
Software utilities for generating and manipulating 2D contours and 3D surfaces include 3D Slicer (Pieper et al., 2006; Gering et al., 1999) and ITK-SNAP (Yushkevich et 75 al., 2006), and software packages including VTK software system available from Kitware, Inc. of Clifton Park, N.Y. (See Schroeder et al., The Visualization Toolkit, 4th Ed., Kitware, 2006), and Insight Registration and Segmentation ToolKit (ITK, Ibanez et al., 2005), the entire disclosures of each of which are incorporated herein by reference.
Three-dimensional (3D) surfaces are typically generated based on contour data corresponding to many two-dimensional (2D) images. Generally speaking, a contour is a set of points that identifies or delineates a portion or segment of an image that corresponds to an object in the image. Each contour separates or segments an object from the remainder of the image. Contours may be generated by computer vision (e.g. by edge detection software), manually (e.g. by a person using a marker to draw edges on an image), or any combination of the two (e.g. by a person using computer-assisted segmentation or contouring software).
An exemplary system may be configured to (1) capture many images of an object of interest from many different viewpoints, (2) perform an automated segmentation process to automatically generate contours that define the object of interest, and (3) generate a 3D surface representative of the object of interest. A 3D surface may be represented by one or more radial basis functions, each centered on a constraint point. Thus, a 3D surface may be defined by a plurality of constraint points.
As an arbitrary example, a system may be configured to capture 100 2D images in each of the coronal, sagittal, and transverse planes, for a total of 300 two-dimensional images. The exemplary system could then automatically generate contours for each of the 2D images using a segmentation process (such as the exemplary segmentation processes disclosed in the cross-referenced applications), and then use the generated contours to create a 3D object representative of the anatomical structure.
Automated or computer generated contouring and segmentation of medical images frequently results in erroneous contours that do not accurately reflect the shape of the anatomical structure shown in the underlying images. Errors may be more prevalent where an original image set suffers from low contrast, noise, or nonstationary intensities.
Furthermore, manual contouring and computer-assisted contouring based on user-input may also result in contours having mistakes that would benefit from further manual editing. For example, a more experienced user may wish to modify erroneous contours created by a less experienced user.
Errors in contours fall into two general categories: under-segmentation and over-segmentation. With under-segmentation, only a first portion of an object (e.g. anatomical structure) is correctly identified by the contour, while a second portion of the object is incorrectly excluded. In the case of over-segmentation, extraneous portions of an image are incorrectly identified by the contour as part of the object (e.g. anatomical structure).
Thus, it is desirable to provide a user with the ability to manually edit contours to correct various mistakes and errors in an existing contour. With existing contour editing software, a user supplies a 2D “edit contour” for an object of interest (e.g. anatomical structure) for one or more 2D images. The user-supplied edit contour data is indicative of an edited or corrected contour for the object.
Due to the large number of underlying images and contours that may be involved, it is preferable that contour editing software not require the user to create edit contours for all of the underlying images. It is preferable to allow the user to modify only a subset of the underlying 2D contours (e.g. based on user selection of the viewpoint or viewpoints in which the error is most clearly visible), and to provide software for correcting a 3D surface shape representative of an object based on the received edit contours.
The inventor has identified various problems that arise in the process of modifying contours and surfaces. For example, one problem is the difficulty inherent in deciding which pre-existing constraint points for a pre-existing surface should be eliminated. For example, when an object of interest has been under-segmented, the pre-existing surface will be too small. The received edit contours will thus correspond to a new 3D surface that is larger than the pre-existing surface. Thus, an interface between the pre-existing 3D surface and the new 3D surface may exist, such as a concave deformity. In the case of over-segmentation, the interface may be a convex deformity. It will be understood that any combination of under-segmentation and over-segmentation may occur for a given object of interest. E.g., one portion of an object may be over-segmented, while another portion is under-segmented, as is known in the art. Accordingly, multiple edit contours may be received for a single object in a single image, each edit contour corresponding to a different segmentation error.
Embodiments disclosed herein are directed to correcting errors in one or more pre-existing contour, pre-existing surface, and/or pre-existing set of constraint points. The pre-existing contour(s), pre-existing constraint points, and pre-existing surface(s) may have been automatically or manually generated. Embodiments disclosed herein are directed to modifying a pre-existing three-dimensional surface and/or set of pre-existing constraint points based on one or more received edit contours. Embodiments disclosed herein are directed to creating a new three-dimensional surface and/or set of constraint points based on one or more received edit contours. Embodiments are disclosed for correcting both under-segmentation and over-segmentation.
Embodiments disclosed herein use data corresponding to the received edit contours to selectively eliminate pre-existing constraint points on a pre-existing 3D surface.
These and other features and advantages of the present invention are disclosed herein and will be understood by those having ordinary skill in the art upon review of the description and figures hereinafter.